The GREC Challenges 2010: Overview and Evaluation Results
نویسندگان
چکیده
There were three GREC Tasks at Generation Challenges 2010: GREC-NER required participating systems to identify all people references in texts; for GRECNEG, systems selected coreference chains for all people entities in texts; and GRECFull combined the NER and NEG tasks, i.e. systems identified and, if appropriate, replaced references to people in texts. Five teams submitted 10 systems in total, and we additionally created baseline systems for each task. Systems were evaluated automatically using a range of intrinsic metrics. In addition, systems were assessed by human judges using preference strength judgements. This report presents the evaluation results, along with descriptions of the three GREC tasks, the evaluation methods, and the participating systems.
منابع مشابه
Generating Referring Expressions in Context: The GREC Task Evaluation Challenges
Until recently, referring expression generation (reg) research focused on the task of selecting the semantic content of definite mentions of listener-familiar discourse entities. In the grec research programme we have been interested in a version of the reg problem definition that is (i) grounded within discourse context, (ii) embedded within an application context, and (iii) informed by natura...
متن کاملThe GREC Main Subject Reference Generation Challenge 2009: Overview and Evaluation Results
The GREC-MSR Task at Generation Challenges 2009 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted one system each, and we additionally created four baseline systems. Systems were tested automatically using existing intrinsic metrics. We also evaluated systems extrinsically by applying core...
متن کاملThe GREC Named Entity Generation Challenge 2009: Overview and Evaluation Results
The GREC-NEG Task at Generation Challenges 2009 required participating systems to select coreference chains for all people entities mentioned in short encyclopaedic texts about people collected from Wikipedia. Three teams submitted six systems in total, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evalu...
متن کاملUDel: Named Entity Recognition and Reference Regeneration from Surface Text
This report describes the methods and results of a system developed for the GREC Named Entity Recognition and GREC Named Entity Regeneration Challenges 2010. We explain our process of automatically annotating surface text, as well as how we use this output to select improved referring expressions for named entities.
متن کاملThe GREC Challenge 2008: Overview and Evaluation Results
The GREC Task at REG’08 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted a total of 6 systems, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference ...
متن کامل